feat(file): search workspace files by regular expression - #7370

The File block's Search operation read its query as literal text. It now reads it as a line-oriented regular expression by default, with a Match setting on the block to go back to verbatim text.

Why this needed no migration

The segment store already indexes content with gin_trgm_ops. pg_trgm extracts trigrams from a regex source too, not just a LIKE pattern, so ~ / ~* plan as a bitmap index scan exactly like LIKE / ILIKE. Measured on a 200k-row table with that index:

No new index, no schema change, no backfill.

Shape

One compiled pattern owns every mode-specific decision — how PostgreSQL matches a segment, whether the segment must hold a whole logical line, and where the match sits inside it — so the repository builds one query shape and the preview renderer one preview shape.

regex.ts    parses the supported subset, computes the guaranteed literal run, rewrites \b → \y
pattern.ts  compileFileSearchPattern(query, mode) → { sqlPattern, caseSensitive,
                                                      literalText, wholeLineOnly, findMatchRange }
repository  builds one query from it
text.ts     renders one preview from it

Compilation happens in the application use case, not the route adapter, so any future surface (v2 API, Copilot) gets identical semantics and the same actionable validation error.

Supported syntax

Deliberately the intersection of PostgreSQL ARE and JavaScript RegExp, because the same source drives both the indexed ~ / ~* predicate and the client-side match location a preview centres on. Anything the two engines read differently is rejected by name rather than silently reinterpreted.

Supported: . * + ? {n,m} and lazy forms, [a-z] / [^0-9], \d \w \s \D \W \S, |, (...) / (?:...), ^ $, \b.

Rejected with a message naming the alternative: lookaround, backreferences, named groups, (?i), \p{...}, POSIX classes, PostgreSQL-only escapes.

\b is a backspace in PostgreSQL, not a word boundary, so it is rewritten to \y on the way out — the one dialect gap a user would otherwise hit blind. No multiline: a match lies within one line.

Not dangerous — four independent layers

1. Literal-run gate. A pattern must contain 3 consecutive literal characters every match will include. pg_trgm can index nothing shorter, and an unextractable pattern plans as a Parallel Seq Scan across every workspace's segments. error \d+ passes; \d{4}-\d{2}-\d{2} is rejected with a message that says why.
2. new RegExp proves it compiles in JavaScript.
3. PostgreSQL proves it compiles in ARE — 2201B becomes a 400, never a 500.
4. statement_timeout + lock_timeout bound the read, mirroring lib/table/planner.ts. This layer covers exact matching too, closing a pre-existing hole: a punctuation-only or non-ASCII literal query (-->, CJK) has always produced zero trigrams and reached the same cross-tenant scan.

The agent surface

mode is a builder setting, withheld from the model like maxResults — the agent writes the query, the block decides how it is read.

That makes the syntax documentation load-bearing, so toolEnrichment swaps the declared syntax for the active mode's. The two readings disagree on every metacharacter: a regex sent to a block set to exact matching would otherwise be searched for verbatim and silently find nothing. Verified for both modes and for a block saved before the field existed (falls through to the default mode's text).

Verified against a real PostgreSQL 17

The unit tests mock @sim/db, so they never render the SQL the repository emits. A scratch-database harness runs the real thing:

• 14 behavioural checks end to end — exact mode unchanged, alternation, classes, metacharacter-blind smart case, \b → \y, anchors bound to a whole line, split-line previews centred on the match, cross-workspace isolation, guards not leaking past the transaction
• Live regex search over 150,012 segments in 14ms; 6/6 representative patterns reach the trigram index
• The guard cutting a deliberately expensive 12s pattern at 10.08s into Search timed out. Narrow the search…

That harness caught a bug the unit tests could not: drizzle wraps a query failure in DrizzleQueryError with no code, and the driver's SQLSTATE sits one level down on cause. The first version read the top level, so the timeout would have surfaced as a generic 500 with no actionable text.

Behaviour change

Regex is the default at every layer, including for a Search block saved before Match existed (no stored mode → regex). A query containing regex metacharacters changes meaning or errors: config.json also matches configXjson, handleRequest( fails with "Unclosed (", price: $100 never matches. Fix per block is flipping Match to Exact match.

Note that regex mode is not a strict superset — the trigram gate rejects some plain-text queries exact mode accepts, e.g. 1.2.3 has no 3-character literal run.

Checks

bun run test in apps/sim: 39,472 passed, 0 failures. tsc --noEmit clean, biome clean, check:api-validation passed. All re-run on this branch's base (latest staging) after bun install, since the base moved 5 commits including a deps and a test-sharding change.

🤖 Generated with Claude Code